Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add python 3.8 support and deprecate python 3.5 #428

Merged
merged 32 commits into from
Jan 21, 2020

Conversation

mtreinish
Copy link
Member

@mtreinish mtreinish commented Nov 11, 2019

Summary

This commit adds support for running aer under python 3.8. It does
this by adding the trove classifiers to the package metadata and adding
a test job for python 3.8 At the same time this commit starts the
deprecation window for python 3.5. Python 3.5 goes end of life by
upstream python in Sept. 2020. We should give our users running with
python 3.5 (which is about 10% of qiskit users this year based on pypi
data) fair notice that when upstream python stops supporting it we do as
well. Corresponding to this change, terra is going to start raising a
warning when anything from qiskit.* is imported, see:
Qiskit/qiskit#3268

Details and comments

Python's documented branch support status and EoL dates are listed here:
https://devguide.python.org/#branchstatus

Waiting on upstream dependencies

This commit adds support for running aer under python 3.8. It does
this by adding the trove classifiers to the package metadata and adding
a test job for python 3.8  At the same time this commit starts the
deprecation window for python 3.5. Python 3.5 goes end of life by
upstream python in Sept. 2020. We should give our users running with
python 3.5 (which is about 10% of qiskit users this year based on pypi
data) fair notice that when upstream python stops supporting it we do as
well. Corresponding to this change, terra is going to start raising a
warning when anything from qiskit.* is imported, see:
Qiskit/qiskit#3268
@mtreinish mtreinish added the on hold Can not fix yet label Nov 11, 2019
@mtreinish mtreinish removed the on hold Can not fix yet label Nov 12, 2019
@mtreinish
Copy link
Member Author

Hmm, the windows python 3.8 job is failing trying to install scipy from sdist. But, there are publish python 3.8 windows wheels for scipy on pypi: https://pypi.org/project/scipy/#files I wonder why it's not using that.

For whatever reason the scipy package on the windows python 3.8 jobs is
trying to build from sdist. This requires blas and lapack libraries
installed because it is building the package from source. Instead of
relying on pip to install scipy this just switches to using the
conda-forge version of numpy and scipy which are precompiled for windows
and should work fine.
The python 3.8 windows ci job can not install jupyter sphinx because of
an issue with the pywin32 package. Pip is unable to find a compatible
package for this and conda installing it doesn't satisfy the requirement
for pip either. To workaround this issue this adds a version cap on
jupyter-sphinx so we do not try to install it by default in every test
environment. The package is only needed for building documentation and
since it's unlikely that people will be installing docs on python 3.8 at
this point and our ci jobs do not we use this to unblock python 3.8 jobs
on windows temporarily. In the future though we'll have to remove this
when people start using 3.8 as their default interpreter so that
building docs with python 3.8 is feasible.
@mtreinish mtreinish added on hold Can not fix yet and removed on hold Can not fix yet labels Nov 13, 2019
atilag
atilag previously approved these changes Dec 6, 2019
Copy link
Member

@atilag atilag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!
Thanks Matthew! ... will merge once all the deps are ready.

- bash: |
set -e
source activate qiskit-aer-$(Build.BuildNumber)
python -m pip install -v --disable-pip-version-check pip==18
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to stick to pip 18 anymore :) pip >= 19.2.1 (IIRC) has finally fixed the bug with PEP 517.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See: #494

@mtreinish mtreinish mentioned this pull request Jan 17, 2020
13 tasks
@mtreinish
Copy link
Member Author

osqp has python 3.8 binaries published on conda. We're using conda for windows testing anyway so this resolves that dependency. So this should be good to go now (assuming I didn't make a typo that breaks the build).

Copy link
Member

@atilag atilag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tests still failing, seems like the conditional to install osqp on python 3.8 is not working

CHANGELOG.md Outdated Show resolved Hide resolved
@mtreinish
Copy link
Member Author

Hmm, interesting it looks like the second conda install line is not being run at all. The stage runs but it only shows the output to conda install mkl. I'll try switching it to bash and setting trace mode to see if I figure out what's going on more.

The script directive doesn't look like it's every command specified. The
output for the python 3.8 anaconda install step doesn't seem to include
the output for the second conda install command. This commit switches it
to use bash so we can set trace mode to see if we're even running the
command, or if something else is happening.
osqp is only installed as a dependency for cvxpy. So instead of trying
to install osqp and getting pip to see that the dep is already
satisified when installing cvxpy it'll be easier to just install cvxpy
via anaconda instead.
The azure pipelines stages had unecessary steps and duplicated logic.
This commit cleans it up slightly and makes it easier to follow.
Oddly it looks like pip isn't finding osqp when installing the
requirements outside in a separate stage. This commit attempts to fix
the issue by moving the dependency installation into it's own stage.
@mtreinish
Copy link
Member Author

Ok, after shifting things around to do all the conda installs first and then doing pip second as well as tweaking some versions I got everything running successfully on windows too. This if finally good to go.

@mtreinish mtreinish requested a review from atilag January 21, 2020 09:18
@chriseclectic chriseclectic merged commit c745829 into Qiskit:master Jan 21, 2020
chriseclectic pushed a commit to chriseclectic/qiskit-aer that referenced this pull request Jan 23, 2020
* Add python 3.8 support and deprecate python 3.5

This commit adds support for running aer under python 3.8. It does
this by adding the trove classifiers to the package metadata and adding
a test job for python 3.8  At the same time this commit starts the
deprecation window for python 3.5. Python 3.5 goes end of life by
upstream python in Sept. 2020. We should give our users running with
python 3.5 (which is about 10% of qiskit users this year based on pypi
data) fair notice that when upstream python stops supporting it we do as
well. Corresponding to this change, terra is going to start raising a
warning when anything from qiskit.* is imported, see:
Qiskit/qiskit#3268

* Install numpy and scipy from conda forge

For whatever reason the scipy package on the windows python 3.8 jobs is
trying to build from sdist. This requires blas and lapack libraries
installed because it is building the package from source. Instead of
relying on pip to install scipy this just switches to using the
conda-forge version of numpy and scipy which are precompiled for windows
and should work fine.

* Add conda jupyter package to 38 windows job

* Conda install pywin32 too

* Cap jupyter-sphinx version to non python 3.8 jobs

The python 3.8 windows ci job can not install jupyter sphinx because of
an issue with the pywin32 package. Pip is unable to find a compatible
package for this and conda installing it doesn't satisfy the requirement
for pip either. To workaround this issue this adds a version cap on
jupyter-sphinx so we do not try to install it by default in every test
environment. The package is only needed for building documentation and
since it's unlikely that people will be installing docs on python 3.8 at
this point and our ci jobs do not we use this to unblock python 3.8 jobs
on windows temporarily. In the future though we'll have to remove this
when people start using 3.8 as their default interpreter so that
building docs with python 3.8 is feasible.

* Stop trying to rely on conda-forge and just try pip

* Add -v to pip install for debug

* Don't upgrade pip and friends for python 3.8 windows job

* Add osx job to premerge ci too

* Fix typos in travis config

* Install osqp via conda

* Fix typo in changelog

* Switch to bash for conda install on 3.8

The script directive doesn't look like it's every command specified. The
output for the python 3.8 anaconda install step doesn't seem to include
the output for the second conda install command. This commit switches it
to use bash so we can set trace mode to see if we're even running the
command, or if something else is happening.

* Fix variable access

* Break out 3.8 requirement binary stage and switch to cvxpy

osqp is only installed as a dependency for cvxpy. So instead of trying
to install osqp and getting pip to see that the dep is already
satisified when installing cvxpy it'll be easier to just install cvxpy
via anaconda instead.

* Back to osqp cvxpy fails to install

* Tweak stages on azure pipelines slightly

The azure pipelines stages had unecessary steps and duplicated logic.
This commit cleans it up slightly and makes it easier to follow.

* Try splitting out dependency installs

Oddly it looks like pip isn't finding osqp when installing the
requirements outside in a separate stage. This commit attempts to fix
the issue by moving the dependency installation into it's own stage.

* Install more python deps via conda

* Fix bash variable access again
chriseclectic pushed a commit to chriseclectic/qiskit-aer that referenced this pull request Jan 23, 2020
* Add python 3.8 support and deprecate python 3.5

This commit adds support for running aer under python 3.8. It does
this by adding the trove classifiers to the package metadata and adding
a test job for python 3.8  At the same time this commit starts the
deprecation window for python 3.5. Python 3.5 goes end of life by
upstream python in Sept. 2020. We should give our users running with
python 3.5 (which is about 10% of qiskit users this year based on pypi
data) fair notice that when upstream python stops supporting it we do as
well. Corresponding to this change, terra is going to start raising a
warning when anything from qiskit.* is imported, see:
Qiskit/qiskit#3268

* Install numpy and scipy from conda forge

For whatever reason the scipy package on the windows python 3.8 jobs is
trying to build from sdist. This requires blas and lapack libraries
installed because it is building the package from source. Instead of
relying on pip to install scipy this just switches to using the
conda-forge version of numpy and scipy which are precompiled for windows
and should work fine.

* Add conda jupyter package to 38 windows job

* Conda install pywin32 too

* Cap jupyter-sphinx version to non python 3.8 jobs

The python 3.8 windows ci job can not install jupyter sphinx because of
an issue with the pywin32 package. Pip is unable to find a compatible
package for this and conda installing it doesn't satisfy the requirement
for pip either. To workaround this issue this adds a version cap on
jupyter-sphinx so we do not try to install it by default in every test
environment. The package is only needed for building documentation and
since it's unlikely that people will be installing docs on python 3.8 at
this point and our ci jobs do not we use this to unblock python 3.8 jobs
on windows temporarily. In the future though we'll have to remove this
when people start using 3.8 as their default interpreter so that
building docs with python 3.8 is feasible.

* Stop trying to rely on conda-forge and just try pip

* Add -v to pip install for debug

* Don't upgrade pip and friends for python 3.8 windows job

* Add osx job to premerge ci too

* Fix typos in travis config

* Install osqp via conda

* Fix typo in changelog

* Switch to bash for conda install on 3.8

The script directive doesn't look like it's every command specified. The
output for the python 3.8 anaconda install step doesn't seem to include
the output for the second conda install command. This commit switches it
to use bash so we can set trace mode to see if we're even running the
command, or if something else is happening.

* Fix variable access

* Break out 3.8 requirement binary stage and switch to cvxpy

osqp is only installed as a dependency for cvxpy. So instead of trying
to install osqp and getting pip to see that the dep is already
satisified when installing cvxpy it'll be easier to just install cvxpy
via anaconda instead.

* Back to osqp cvxpy fails to install

* Tweak stages on azure pipelines slightly

The azure pipelines stages had unecessary steps and duplicated logic.
This commit cleans it up slightly and makes it easier to follow.

* Try splitting out dependency installs

Oddly it looks like pip isn't finding osqp when installing the
requirements outside in a separate stage. This commit attempts to fix
the issue by moving the dependency installation into it's own stage.

* Install more python deps via conda

* Fix bash variable access again
@mtreinish mtreinish deleted the deprecate-3.5 branch February 22, 2020 12:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants